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Abstract 

This article explores how online distance learning tutors working within a higher education 
context may be supported in their professional development through participation on an 
institutional peer observation programme. Drawing on the reflections of participants from two 
cohorts (2011 and 2012) at the University of York, the article reviews the learning outcomes 
arising from the peer observation process and investigates the necessary conditions for fostering 
critical reflection on practice between tutors. Feedback from participants highlighted the 
importance of investing time in relationship-building at the outset of the process - a necessary 
first step before free-ranging and critical exchanges can flourish between partners. Pairings which 
invested time at this stage appeared to develop greater levels of trust, enabling them to probe 
their tutoring practices in a more critical way and explore areas of perceived weakness as well as 
tackling emergent themes in their dialogue. These observations have informed the development 
of a guidance framework for peer exchange at the University of York, which is summarised in the 
concluding part of this paper. 

Keywords: peer observation; online tutoring; peer review 

Introduction 

Fully online course delivery is growing in importance across the UK higher education sector: 
over 2,800 FIE courses are now offered online and the numbers are rising (Baxter, 2012). The 
emergence of MOOCs has captured the attention of government (QAA, 2013) and university 
senior managers (Gaebel et al., 2014; Nutbeam, 2013) alike, as a way of engaging new types of 
learners and directing them to accredited programmes, and yet questions remain around the 
impact of the pedagogies employed for online learning and the skill-set of tutors to support 
online learners effectively (Online Learning Task Force, 2011). Gallardo, Fleiser and Nicholson 
(2011) note that the online environment offers a fresh and demanding context in which to 
manage interactions with learners based on more flexible and diverse engagement methods, and 
this requires different skills and values (Baxter, 2012; Kirkwood & Price, 2012). 

The challenges of designing and supporting effective online learning have promoted calls for the 
‘unlearning’ of established classroom pedagogies (McWilliam, 2005), with tutors encouraged to 
develop new strategies for engaging with learners and as well as their own online teaching 
persona (Flarper & Nicolson, 2013). Bennett and Marsh (2002) see this as an imperative, given 
that that “the majority of tutors new to online tutoring do not have that background of online 
learning experience upon which to draw” (p. 15). The transition to online tutoring requires staff to 
develop pedagogical and managerial skills, as much as technical skills in support of their online 
learners (Bowskill et al., 2000; Flarper & Nicholson, 2013). Flowever, conventional practice has 
tended to neglect these measures, with tutors commonly faced with having to develop their e- 
teaching skills without sufficient institutional support (Flauck & Stickler, 2006). 
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How then should institutions tackle this deficit in skills and values, supporting new and existing 
tutors in their professional development? In recent years there has been growing interest in 
applying peer observation techniques to online teaching and tutoring activities as a way of 
addressing this challenge (e.g. Bennett & Barp, 2008; Bennett & Santy, 2009; Goldsmith & 
Rogers-Ward, 2003; Harper & Nicolson, 2013; Swinglehurst, Russell & Greenhalgh, 2008; 
Tonkin & Baker, 2003). Indeed a variety of methods have been developed across the UK higher 
education sector to support peer review of this kind, with a primary focus on support for 
distance learning delivery. The ‘Peer-to-peer Reflection on Pedagogical Practice’ (PROPP) model 
developed at University College London is one such example, providing a framework for 
individual tutors to come together to discuss a problem, issue or a project from a course and use 
this material as a basis for reflection on course design and delivery methods (Swinglehurst et al., 
2008). The Universities of Hull and Staffordshire have developed their own collaborative model 
of peer observation of online practice for online tutors (COOLAID), engaging tutors across both 
institutions (Bennett, Lee, Lynch & Howard, 2010). 

Both the PROPP and COOLAID programmes mark a departure from traditional peer 
observation approaches for classroom teaching which have focused on quality improvements and 
the evaluation of teaching performance; they are designed instead to offer learning opportunities 
for both observer and observee in sharing practice and pedagogic approaches to online tutoring, 
drawing inspiration from Gosling’s ‘Peer Review’ model (Gosling, 2002) which emphasises the 
added value of dialogue and the sharing of perspectives for teacher development. Relating peer 
review methods to online tutoring, Bennett and Santy (2009) note the potential for a supported 
dialogue between tutors to offer a ‘window’ on each other’s e-learning practice, providing a 
means by which they can deepen their understanding of the online learning and teaching process 
and develop an awareness of what works in the online environment. It is precisely within this 
context of peer dialogue between colleagues that this research is situated - exploring the 
conditions for effective exchange to take place for distance learning tutors through an 
institutional peer observation programme. Through a review of participants’ feedback, the paper 
discusses both the challenges and opportunities for the development of tutors’ online skills and 
values. 

York's approach to peer observation 

The University of York in the United Kingdom currently offers 17 distance learning 
programmes, almost exclusively for postgraduate students, which are managed locally by 
departmental teams. Programme teams are subject to the quality assurance and management 
oversight of their affiliated departments, but do send representatives to the University’s Distance 
Learning Forum - a body set up in 2007 to foster peer support and disseminate good practice 
between those involved in mnning these programmes. The Forum facilitates the exchange of 
information and discussion of issues of common interest in the mnning of distance learning 
programmes, whilst also serving as a source of advice and expertise to the University’s Teaching 
Committee on distance learning matters, touching on quality enhancement and policy matters. 

Recognising the untapped potential to share innovative teaching and programme delivery 
approaches across the distance learning community whilst also supporting individual tutors in 
their professional development, the University of York’s Distance Learning Fomm took the 
decision in 2011 to launch an institutional peer observation programme as a complement to the 
University’s Peer Support for Teaching provision. The programme was influenced in its design 
by insights from the COOLAID project and related approaches informed by Gosling’s Peer 
Review model (2002). The Forum also drew on the practical experience of three of the distance 
learning programme teams at York, whose own internal observation schemes pre-dated this 
initiative. Peer observation was presented as a development opportunity for the individual tutors 
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involved, rather than as a quality assurance mechanism or way of monitoring performance, 
focusing on the ‘mutual exchange of practice’ (McMahon, Barrett & O’Neill, 2007). Tutors from 
all of the University’s online distance learning programmes were invited to opt in to the 
programme and were free to determine the terms under which they would participate — i.e. the 
development objectives which they would wish to pursue. 

Table 1 below captures the similarities and differences between the University of York’s model 
and COOLAID. Whilst retaining the central purpose of supporting non-judgemental and 
constmctive dialogue on tutoring practice, the York model differs from established peer 
observation frameworks (e.g. Bennett et al., 2010; Carroll & O’Loughlin, 2013) that favour the 
self-selection of peers. Participants who opted in to the York programme were assigned to 
random pairings in the role of observer and observee, bringing colleagues together from different 
disciplinary backgrounds with contrasting levels of tutoring experiences. This was intentional in 
exposing participants to different pedagogic perspectives on tutoring skills, as well as unfamiliar 
programme contexts and technologies (learning platforms), taking them out of their ‘comfort 
zone’. It was also intended to encourage them to focus on tutoring rather than disciplinary issues 
and in this way extend the opportunities for mutual learning within the pairings. Furthermore, 
there was no requirement for reciprocity in the observation process: participants were free to 
determine how and when they would conduct the peer observation - whether this would be a 
reciprocal arrangement (i.e. participants experiencing both roles as observer and observee) or 
one-way arrangement. The only requirement was for participants to assist with the evaluation of 
the programme, by engaging in a reflective interview on their experiences with the coordinator 
upon completion of the process. 


Table 1: York's peer observation model 


Description of York approach 

Features based on COOLAID approach 
(Bennett et al., 2010) 

Features specific to York 

Purpose: 


• 

Engagement in discussion about online 
tutoring practice. 


• 

Non-judgemental and constructive. 


Choice over: 

Choice over: 

• 

Participation (opt-in) 

• role (observer / observee) 

• 

Focus of observation (participant- 

(N.B. no obligation to have own 


ownership of agenda) 

practice reviewed) 

• 

Form of feedback 


• 

Information flow 

No choice over: 

• 

Future action 

• Partner (observer and/or observee) 



Random selection - based on 
matching participants from different 
programmes & subject / disciplinary 
backgrounds 
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Peer observation programme format 

Figure 1 below captures the key steps in the delivery of the peer observation programme. For the 
inaugural programme, an online consultation phase was introduced over a one-week period prior 
to the launch (14 th - 21 st Feb 2011), which aimed to help tutors understand the aims of the 
programme and foster a sense of ownership over the peer observation process. A shared blog 
space was set up for participants to comment on the observation process and the guidance 
documentation that they would be using. Tutors were then blind matched by the programme 
coordinator and invited to make contact with their partner and proceed with the identification of 
objectives and definition of roles as part of their observation ‘contract’. 

For both cohorts, general guidance notes were made available to participants at the beginning of 
the programme on the principles of peer observation, approaches to defining the focus for an 
observation and tips on the delivery of feedback to observees. Practical advice was also provided, 
such as a reminder that observees should notify their students if a ‘live’ teaching site was being 
observed by an external tutor, as opposed to an archived site where a notification to students 
would not be necessary. Participants also received guidance through the provision of a pro forma 
document, which was intended to help participants reflect on their goals for the activity, 
recording their observation arrangements (objectives and observation aims; arrangements for 
observation/information to provide to observer and tips on delivery of feedback). Individuals 
were requested to return the first section of the form to the programme coordinator during the 
spring term and before commencing the observation process, outlining the agreed aims of the 
observation. Partners were left free thereafter to manage the observation process as they saw fit, 
defining their own engagement methods and timelines, with a deadline of the end of the summer 
term set for the completion of the observation process and evaluation of learning outcomes. The 
lack of direction was intentional in giving participants maximum flexibility to manage the 
relationship and balance the observation tasks with their day-to-day tutoring responsibilities. 



Online 

consultation 

•(2011 only) 



Blind 

matching of 
pairs by 
programme 
coordinator 

•February 



Definition of 

observation 

objectives 

•return of pro 
forma (spring 
term) 



Conduct of 
observations 

(summer term) 



Evaluation 

of 

observation 
experience 
& learning 
outcomes 

(end of summer 
term) 






Figure 1. Key steps in the delivery of the peer observation programmes (2011 & 2012) 
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Research methodology 

The research study aimed to explore the necessary conditions for the effective conduct of 
observations for online distance learning tutors- specifically how tutors could be encouraged to 
engage in critical reflection on their tutoring practice through peer exchange and dialogue. As 
part of the investigation into the conditions, the study explored the impact of the blind matching 
process to see how the combinations of pairings worked and why, as well as the range of 
contextual information that should be shared prior to an observation in order to support an 
effective exchange between participants. 

The research approach was designed around participants’ perceptions of their experiences, 
establishing ‘meaning’ from the standpoint of the actors (Cohen & Mannion, 1994), rather than 
through objective measurements of performance or levels of engagement in the peer observation 
programme. Evaluation of learning outcomes from the programme was consequently based on 
interpretive methods through semi-structured interviews with observers and observees after the 
completion of each observation. This research approach was selected to provide an insight into 
tutors’ perceptions of their shared experience, as expressed through their own language and 
reflections (Gillham, 2000). All participants were approached for feedback on their experiences at 
the end of the observation cycle, with tutors invited to discuss their experience in response to six 
stimulus topics: (i) expectations towards the observation process; (ii) reflections on the blind 
matching process; (iii) explanation of how they negotiated the peer observation contract; (iv) 
experience of the observation and feedback processes; (v) lessons learned from the observation 
process and impact on professional development and (vi) key conditions for the observation 
process to work effectively. The interview was conducted, where possible, as a mediated dialogue 
between partners, face-to-face or by telephone. Transcripts of the interviews were shared with 
participants for validation purposes. Participant reflections on the learning outcomes were then 
cross-referenced with the declared objectives for each observation, which had been documented 
in the pro forma completed by participants at the outset of the process, as a sense-checking 
measure as well as a way of reviewing the evolution of the learning exchange for each pairing. 

Interview transcripts for each cohort were then combined together and a qualitative content 
analysis was performed of recorded comments for each of the six discussion topics as part of an 
inductive evaluation approach (Hsieh & Shannon, 2005). This aimed to identify key themes in the 
participant experience and support grounded theory development in relation to the research 
questions. As a final step, summary findings from the content analysis were shared in a short 
report with each cohort of tutors to check for validity - specifically checking that the correct 
inferences had been drawn from the data. 
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Profile of the peer observation participants 

Expressions of interest were received from 12 tutors from three distance learning study 
programmes in 2011, of which 10 tutors went on to participate, and from 16 tutors from five 
programmes in 2013, of which 14 participated. Five tutors participated across both programmes, 
working with a different partner each time, as illustrated in Table 2 below. 


Table 2: Participating tutors in the peer observation programmes (2011 & 2012) 


Distance Learning Programme 

Participating tutors 



2011 peer 

2012 peer 

# of tutors 


observation 

observation 

participating 


programme 

programme 

in 2011 & 




2012 

Lifelong Learning (learning for 
pleasure): 

- Certificate in Creative Writing 

3 



- York 800 course 

- 

1 

- 

Online MSc in Haematopathology 

- 

1 


MA in Teaching English to Young 
Learners 

2 

2 

[1] 

Postgraduate Certificate/ Diploma/ 
Masters in Public Policy and 
Management 

- Public Administration 

- Public Administration in 

International Development 

5 

7 

[4] 

Postgraduate Certificate and 

Diploma in Health Economics for 
Health Care Professionals 


3 


TOTAL 

10 

14 

[51 


Volunteers were drawn from different disciplines and study programmes with contrasting online 
tutoring approaches, including asynchronous group discussion and one-to-one supervision, 
involving a mix of tools and learning platforms (e.g. Moodle and Blackboard) to support their 
students. Table 3 summarises the contrasting online teaching approaches and delivery models for 
the participating programmes from which the tutors were drawn. 
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Table 3: Contrasting teaching and delivery models of the participating distance learning 
programmes 


Distance Learning Programme 

Delivery and teaching mode 

Lifelong Learning (learning for 
pleasure): 

- Certificate in Creative Writing 

- York 800 course 

Online tutored 

Hosted on Blackboard. Fully online (tutored). Students 
complete individual writing tasks. No formal requirement 
for students to engage in asynchronous group 
discussion, although blogs made available for York 800 
course. 

Online MSc in Haematopathology 

Distance Online Support 

Hosted on Blackboard. Fully online (tutored). Students 
are provided with access to resources and a group 
discussion space and also participate in synchronous 
group tutorials 

MA in Teaching English to Young 
Learners 

Distance Online Support 

Hosted on Blackboard. Fully online (tutored) for two 
weeks then students use published materials on CD for 
the rest of their study (tutored). Students not required to 
contribute to discussion space. 

Postgraduate Certificate/ Diploma/ 
Masters in Public Policy and 
Management 

- Public Administration 

- Public Administration in 

International Development 

Online discussion-based 

Hosted on Moodle. Fully online (tutored). Course 
materials and discussion space supported online. 

Students are required to participate in weekly 
asynchronous group discussions, which represent a key 
element of the course, preparing them for the end of 
module assignment. 

Postgraduate Certificate and 

Diploma in Health Economics for 
Health Care Professionals 

Distance Online Support 

Hosted on Blackboard. Students use the VLE to submit 
assessments and have optional use of group discussion 
boards to raise issues with their peers and tutor. 


Differences were also evident in the levels of tutoring experience which participants brought to 
the peer observation programmes, ranging from tutors with eight — nine years’ experience to one 
year or less. One of the tutors had only taught two terms at York for the online programme, but 
had face-to-face teaching experience to draw on. A common characteristic shared by all though 
was their self-confidence and willingness to share practice with peers outside their programme. 
As one participant noted in discussing her rationale for involvement on the programme: 

I am confident about my teaching style but not clear if this is working online -I wanted to 
find out. ” (Lifelong Learning tutor - 2011 programmej 

Another of the less experienced tutors commented on the desire to “ improve my abilities: 

“I was not concerned about exposing myself - maybe yes if this had been in my first tern’s 
teaching — but I was confident enough to do it”. (Public Poliy <& Management tutor - 
2012 programme) 
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The fact that the programme was non-compulsory reassured some participants and helped them 
to focus on it as a staff development opportunity, rather than as a competency measurement: 

“I felt in control of the process and quite relaxed. ” (Public Policy <& Management tutor - 
2011 programmej 

Indeed features of the peer observation process were already established as part of working 
practice across some of the distance learning programme teams. Public Policy and Management 
and MA in Teaching English to Young Learners tutors were familiar with peer observation as a 
staff development tool, having engaged with colleagues on their own programme in the 
observation of their tutoring. The opportunity to work with someone outside the programme 
offered fresh perspectives as well as learning opportunities on how another distance learning 
programme works. 

“I was excited by working with an external. We have done our own peer observation 
internally with all following the same approach - this is an opportunity for another pair of 
eyes although it was slightly nerve wracking. ” (Public Polity <& Management tutor— 2011 
programme) 

Research findings 

All 24 participants across both programmes completed and returned the pro forma document, 
capturing their objectives for the observation process, and exit interviews were also held with all 
participants - conducted mainly face-to-face or by phone as individual or mediated discussion 
sessions between the observation partners and the programme coordinator (the author of this 
paper). Two interviews were conducted by email correspondence for tutors posted overseas in 
different time zones, who were unable to join a mediated discussion. Feedback from both 
cohorts was then cross-referenced against pro forma objectives and then the transcripts were 
combined and analysed in relation to the key research themes, as discussed below. 


Blind matching process 

Feedback from the exit interviews revealed that the blind matching approach was positively 
received by participants on both programmes. Contrary to findings from previous studies (e.g. 
Carroll & O’Loughlin, 2013) which have highlighted the importance of participant control in peer 
matching as a way of building confidence, York participants appeared happy to relinquish 
control, expressing no strong preferences over the identity of their partner. An external 
perspective promised a more objective review of tutoring practices, with no preconceptions 
about the programme or individual’s tutoring style: 

“It brought objectivity to the process. My partner didn’t know me or my tutoring style.” 

(Public Polity <& Management tutor - 2011 programme) 

“It allows a certain level of objectivity. You can stand back outside of your course without any 
link with the set-up and comment. ” (MA in Teaching English to Young Learners tutor - 
2011 programme) 

Pedagogic values and levels of tutoring experience were not viewed as critical factors in 
determining an effective ‘match’. 


“Sharing the same pedagogic approach can shut down opportunities to learn. With blind 
matching you can get a nugget from watching anyone. ” (Lifelong Learning tutor -2011 
programme) 
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“It’s not too much of an issue whether your partner has a little or a lot of tutoring 
experience. ” (Public Polity <& Management tutor - 2012 programme) 

Participants noted that the natural inclination for many would be to choose someone from a 
similar discipline, but this might restrict the scope of the learning on offer between partners - 
specifically the contrasting perspectives on pedagogy, terminology and teaching practice. The 
blind matching process acted as an encourager for participants to go beyond their comfort zone 
and work with tutors from different disciplines and pedagogic outlooks- in turn this ensured that 
the focus of the observation was on tutoring practice rather than teaching content. 

‘The major objection to blind matching is that you can’t comment on content and specifics - 
you can ’t comment on the real content of courses. Put the purpose is on how teaching and 
tutoring are done. It’s a good idea to get a mix of academic backgrounds as you can then hope 
to avoid discussing details (of a course). The internal peer observation discussion starts on 
tutoring but soon shifts to the details — the content and not tutoring style. ” (Public Polity <& 
Management tutor — 2011 programme) 

“You can get bogged down in your discipline —you should be focusing on teaching practice 
rather than teaching content...by stripping out the content I could focus on the observation 
issues” (MA in TeachingLnglish to Young Learners tutor — 2011 programme) 

Participants acknowledged though that to make the blind pairing work, they needed to seek out 
common ground between them to establish an effective working relationship and rapport- a 
prerequisite for constructive dialogue (Gosling, 2002; Shortland, 2007 & 2010). This led one 
participant to argue that it is more important to match individuals according to personality - to 
ensure a compatible working relationship - than to match around discipline or pedagogic 
outlook. 

The effectiveness of the observation also depended upon clarifying the course context up front - 
teasing out the information that the observer requires to make sense of the online activities under 
review. This encompasses the ‘rules of engagement’ for the course: i.e. whether participation in 
online activities is a requirement for the course; what’s visible online in terms of learning and 
tutoring support provision and what’s happening offline - e.g. telephone contact with students. 
The gaps in knowledge and assumptions about tutoring styles needed to be addressed up front or 
pursued through clarifying questions as the observation progresses: 

“..that aspect of not knowing the course —you have to ask a lot of questions about the 
background of the course. For one course there was a lot of email / telephone traffic between 
the student and tutor which was not apparent through simply observing the online discussion. 

There were therefore gaps in our knowledge when revieiving the online discussion. Put if we 
repeated the process we would have that knowledge. Could do with that knowledge (context of 
course) upfront. ” (Public Polity <& Management tutor — 2011 programme) 
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Emergent working models for peer observation 

Bennett et al. (2010) in their COOLAID approach describe peer observation as a structured 
process, requiring participants to tackle three phases of engagement, namely: 

1. Preparation: scene setting; establishing roles, discussing objectives for the peer 
observation and expectations on what will be observe and by when - recorded in a 
‘contract’) 

2. Observation: review of tutoring practice and recording of notes in line with ‘contract’ 

3. Discussion: delivery of feedback to observee; reflection and discussion; points for future 
in professional development. 

Feedback from the York participants revealed a great deal of variation in how pairs tackled these 
phases of engagement, with three working models emerging, namely: 


• Model /:face-to-face contract negotiation and discussion of feedback 

• Model 2: contract negotiation at a distance with interactive (oral & written) feedback and 
discussion 

• Model 3\ contract negotiation at a distance with written (one-off) feedback 

Figure 2 below captures the key features of these peer engagement models, summarising how 
participants prepared for and conducted the peer observation and exchange of feedback with 
their partner. 


Model Preparation Observation & Discussion 



online observation 

feedback through f2f 
meeting 

reports exchanged after 
meeting 


online observation 

delivery of notes to 
partner 

discussion of feedback 
by skype 


online observation 

delivery of written 
report to partner 


Figure 2. Emergent working models for the conduct of the peer observation process 


Interestingly, all pairings bar one interpreted the peer observation process as an opportunity for 
mutual sharing of their practice (observing and being observed), with one participant postponing 
her observation to a later date when she had begun tutoring. The choice of engagement model 
varied a great though per pairing and appeared to be determined by a range of variables including 
the geographical location of tutors (location and time zones) and time constraints (i.e. the 
personal time investment that could be devoted to the various stages of the peer observation 
process). For example, two of the pairings involved in the 2012 programme were working in 
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different time zones and countries, which complicated opportunities for synchronous 
communication, leading to an asynchronous working relationship. 

Two pairings from the 2011 programme were able to meet in person (Model 1) and used an 
initial meeting to define the scope of the observation and to surface a range of shared concerns, 
which they then worked up into an agreed set of objectives for the observation - of which there 
was a close overlap of personal interests between partners. A follow-up face-to-face meeting 
provided an opportunity to discuss observation findings and explore their significance for 
tutoring practice, before a summary report was exchanged. This reflected a collaborative and 
conversational approach to the discussion of aims and findings. In contrast, participants working 
at a distance (Models 2 and 3) tended to focus on their own context when identifying objectives 
for the observation, rather than looking for common areas for exploration with their partners 
first, as a precursor to completing the forms. Where meetings were arranged by Skype between 
partners (Model 2), they tended to focus more on introducing each other and addressing 
expectations on how the observation process would work, rather than exploring common ground 
in tutoring values. The key difference between these relationships ‘at a distance’ was in the 
delivery of feedback, with Model 2 pairings discussing outcomes from the observation after the 
exchange of reports, whereas Model 3 pairings simply exchanged feedback reports. 

Learning outcomes from the peer observation process 

The exit interviews revealed different perceptions from participants on the criteria for an 
effective peer observation experience, with perspectives shaped by the working model that 
individuals had agreed on with their partner. For Model 1 pairings the contract objectives and 
agreed focus for the observation of tutoring practice represented a starting point for a more 
wide-ranging discussion of pedagogic practices, addressing differences in tutoring techniques and 
the language and semantics used to describe their respective approaches. The rapport developed 
by Model 1 pairings at the outset of this process enabled them to identify common ground or 
‘closer pedagogic proximity’, providing a platform from which to develop a rich learning 
exchange. 

‘There will be more learning points if there is closer pedagogic proximity between observer and 
observee. ” (Public Polity <& Management tutor— Model 1) 

Model 2 and 3 pairings tended to be more transactional in their outlook, valuing a clear return on 
their time investment in the peer observation process. In the eyes of these participants a 
successful exchange was largely determined by the degree to which partners honoured the time 
commitments bound up in the initial contract in conducting the observation and delivering timely 
and constructive feedback in relation to the objectives that had been set. A successful exchange 
could be measured by the number of ideas that tutors could apply to their own practice; one 
participant assessed the value of the relationship in terms of a simple costs/benefits analysis of 
the ‘take-aways’ matched against the time investment in engaging with the peer observation 
process. 

Valuable insights on tutoring practices were derived from different sources. The ‘learning by 
discovery’ process in reviewing another course site exposed participants to different tutoring 
models. Typically participants noted that they had observed a specific technique or activity in 
their partner’s course which they would like to develop in their own course. 

T especially liked the ivay that you introduced the links with some hint of what the viewer / 
reader would find there. I am thinking about ways to use this approach with introducing 
some of the optional readings for each unit in Moo die... ” (Public Polity <& Management 
tutor — Model 1) 
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The observation process for many was also helpful in validating their tutoring approach, offering 
encouragement to continue along similar lines. The observation process for one Health 
Economics tutor reinforced the value of the discussion rules that had been established in their 
own programme in establishing etiquette for how students and tutors used their discussion space. 
For a Public Policy and Management tutor (Model 3), the affirmative feedback that was received 
“reinforced and gave confidence to do what we would he going to do anyway”. 

Feedback from participants also indicated that learning was not restricted to the predetermined 
focus and agreed aims, but was also experienced incidentally through general observation of 
differences in programmes delivery and ways of doing things. The process of conducting the 
observation with the opportunity to comment on another tutor’s course was as valuable as the 
feedback received from an external observer, stimulating reflection on one’s own practice: 

“The asynchronous exchange forced me to think about why we do things the way we do” 

(Public Polity <&Management tutor- Model2) 

Whilst these outcomes were commonly reported across all observation pairings, distinctive 
benefits were cited by participants with a well-developed personal rapport - specifically Model 1 
participants and Model 2 tutors who had been blind-matched with a colleague with whom they 
had some prior familiarity. Engagement between these colleagues tended to be more free-flowing 

- a continuing dialogue rather than a self-contained and clinical observation of practice and 
delivery of feedback to an unfamiliar partner. Tutors could seek out clarifications on specific 
aspects of the courses that they were reviewing as the observation process unfolded, exploring 
how the context of the course influenced the tutoring approach that had been adopted. This 
provided a frame of reference from which more challenging observations could be made on 
tutoring practice: 

“X noticed a lack of peer work in the module. I am quite busy in student-led discussion. I 
will back off more and change my behaviour. We are looking at peer review assessment across 
the programme team and I am going to bump this up some more. ” (Public Poliy <& 
Management tutor — Model 2) 

For pairings following a more transactional mode of engagement, discussion tended to be less 
spontaneous, with perceived barriers to what could be conveyed to a partner. Delivering critical 
feedback was viewed by some Model 2 and 3 participants as challenging, with observers 
admitting a bias towards reporting positive things in their feedback. As one participant admitted 

- “if I had seen bad practice, I would have probably struggled to tell them”. Other participants noted that 
they stuck closely to the objectives that they had been given by their partner and did not go 
further, or reverted to “neutral feedback”, shelving closely to the pre-set objectives for the 
observation, without negotiating challenging or new areas: 

“It would have been easier to deliver feedback if a relationship had been established. Instead I 
gave neutral feedback, as you would to students, as you don’t know if you will be 
misunderstood.. .1 would have felt reluctan t to give negative feedback — say negative things in 
a direct manner. I would have hidden comments in a more positive statement. ” (Model 3 
Public Poliy <& Management tutor) 

In contrast, a pair of tutors who met face-to-face during the contract negotiation and delivery of 
feedback phases (Model 1) reported that they were able to probe areas of weakness, drawing on 
the relationship and rapport that they had built up. The observer homed in on a suspected area of 
weakness that the observee had wanted to be reviewed as part of the observation process: 
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‘We were able to negotiate criticism — it boils down to personality and trust. You need to be 
undefended in receiving criticism. It was easier to negotiate face-to-face. If we had done this 
online, we could have been constructively critical, but it would have been a different 
experience. ” (Model 1 Lifelong Yearning tutor) 

Delivery of critical feedback was conveyed through questions or suggestions for alternative 
modes (alternative ways of doing things - “haveyou thought about“on our course we did this.. .”), but 
participants noted that care was needed in arriving at critical remarks, given the risk of false 
assumptions based on incomplete information about the course that they were observing. 
Observers needed to be aware of what they couldn’t see - i.e. the levels of student-tutor 
interaction which were taking place outside the observable learning space by telephone or email. 

Some participants used the technique of asking focused questions to explore why their partner’s 
course adopted a particular approach, and through this approach they were able to suggest 
alternative tutoring techniques: 

“I noticed that the private study area was very underused and I was wondering about ways in 
which their attention could be focused there, to support struggling students. Would it be useful 
to place the definitions of terms in this area... ” (Model 3 Lifelong Yearning tutor) 

Discussion 

The evidence from the exit interviews for both peer observation cohorts highlights the 
importance of relationship building at the outset of the process. One participant described the 
building of trust as cmcial “becauseyou are letting someone in ” to observe your practice. This appears 
to be a necessary first step to help create the conditions for a free-ranging and critical relationship 
to develop and one that has the potential to extend beyond the boundaries of the formal 
observation period. The findings show that Model 1 pairings which engaged in negotiation and 
discussion appeared to develop the greatest levels of trust, which enabled them to probe tutoring 
practice in a more critical way, exploring areas of perceived weakness as well as tackling emergent 
themes, moving beyond the agreed objectives. The delivery of critical feedback and attention to 
sensitive areas were reported as being easier to negotiate ‘in person’, where a rapport and shared 
understanding has been established, prompting reflection on practice and declared changes in 
future tutoring behaviour. This also applied to Model 2 pairings, who serendipitously had found 
themselves matched with colleagues with whom they had some prior familiarity, which enabled a 
more personal relationship to develop online. Pairings which worked at a distance without that 
level of familiarity tended to develop their own individual objectives and then adhere more 
closely to the contract, focusing on predetermined themes with objective comments, with limited 
evidence of critical feedback in their exchanges. Exchanges between partners working at a 
distance typically focused on clarification on the context of courses under observation and the 
rules of student engagement, rather than a broader discussion of tutoring principles. The 
influence of physical distance on working relationships and online behaviour is summed up 
neatly in the following comment: 

“Being physically separate leads to individuals finning their own objectives; physical 
proximity may lead to shared values and objectives. ” (Public Policy and Management tutor — 

Model 1) 

The findings from this study therefore concur with previous research on peer observation (e.g. 
Donnelly, 2007; Gosling, 2002; Shortland, 2007) in highlighting the importance of trust in 
creating a conducive environment for open discussion, exploration of issues and risk-taking. This 
raises an important question of how, in the absence of face-to-face contact or a prior relationship 
to draw upon, a bond may be generated between partners to support an effective online 
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exchange of practice. Is there a way of supporting online tutors who are working in different time 
zones and physical locations, so that they may approach the observation process in an 
undefended way, ready to open up their practice to the scrutiny of an unfamiliar partner? The 
limited literature that exists on peer observation for online participants is less helpful in providing 
answers to this question. 

The feedback from Model 2 and 3 pairings in this study perhaps offers useful insights to address 
this challenge. The feedback suggests that the development of critical online relationships for 
tutors working at a distance is unlikely to occur without some form of stmctured relationship 
development being embedded within the observation process. 

“Encourage partners to meet (face-to-face or Skjpe with video) before the process begins to get 
to know each other, and meet afterwards to discuss outcomes. ” (Health Economics tutor - 
Model 2) 

We may infer from this feedback that the engagement of tutors in a preliminary relationship¬ 
forming activity at the outset of the process may help partners to develop a closer bond, 
identifying common interests and shared objectives. Combining this preliminary step with a 
closure activity based on an interactive feedback process, rather than purely textual delivery of 
observations to a partner, may also encourage tutors to discuss emergent themes in tutoring 
practice and engage in more critical exchanges - sharing aspects of their practice which have not 
proven to be effective and using their partner as a sounding board to reflect on why that has been 
the case. Taking this feedback into account, a set of guidelines has been developed to give tutors 
who are new to peer observation a steer on the expected actions and phases of engagement with 
their partners. These guidelines are summarised in Table 4 below: 


European Journal of Open, Distance and e-Learning - Vol. 18 / No. 1 
ISSN 1027-5207 
© 2015 EDEN 


48 



Peer Observation for Online Distance Learning Tutors: Creating the Conditions for Effective Peer Exchange 

Richard Walker 


Table 4: Managing peer observation online - guidelines for participant engagement 


Key steps 

Activity / method 

Timescale 

Step 1: 

Preparing for 
the observation 

(i) Get to know your partner (preferably ‘real time’ - e.g. f2f or 
Skype call), sharing information on: 

- your tutoring approach 

- context of your online course 

- motivation for engaging in peer observation 
identifying common ground & shared interests 

(ii) With your partner: define objectives and roles for peer 
observation, including: 

- module / units forming basis of observation 

- access to online module /unit & context information on course 
(what is online and what is not visible) and what is expected of 
students 

- ethical guidelines on what is / is not meant to be observed - 
particularly for ‘live’ courses 

Completed 
within 2-3 
weeks of 
matching 

Step 2: 

Conducting the 
observation 

(i) Visit a ‘live’ or archived module or unit 

(ii) Observe activity and note-taking in response to observee’s 
questions 

(iii) Broader note-taking and commentary on module / unit 

Conducted 

over a 
period of a 
month for 
‘live’ 

courses. An 
archived 
course will 
be 

considerably 

shorter 

Step 3: 
Exchanging 
feedback with 
your partner 

(i) Communicate feedback through textual exchange (e.g. by 
email) 

(ii) Interactive exchange (preferably real-time - e.g. f2f or 
Skype), discussing the outcomes and clarifying any 
uncertainties. 

Steps (i) and (ii) may be reversed in order, with textual 
feedback serving as record of discussion points from 
interactive exchange 

Completed 

2/3 weeks 
after 

completion 
of the 

observation 

Step 4: 

Evaluating your 
peer 

observation 

experience 

(i) Personal evaluation of learning outcomes from peer 
observation process and consequences for tutoring practice. 

(ii) Identification of actions to be carried forward - ‘change 
agenda’ for personal exploration or for discussion with 
programme team members 

Completed 

immediately 

after 

exchange of 
feedback 
with partner 
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Conclusion 

In drawing conclusions from this study, we should acknowledge the limitations of the research 
evidence that has been generated, which is derived from a small population of self-selecting and 
largely experienced distance learning tutors; some tutors were indeed already sympathetic to the 
aims of peer observation through participation in schemes within their own programme teams. 
Unsurprisingly the evidence confirms the findings reported in other studies (Bennett et al., 2010; 
Goldsmith & Rogers-Ward, 2003; Harper & Nicolson, 2013) that online peer observation may 
contribute important learning benefits for online tutors and serve as a vehicle for professional 
development. Learning benefits were recorded by all participants across the two peer observation 
programmes - irrespective of the working model employed by participants. In contrast though to 
previous studies (e.g. Carroll & O’Loughlin, 2013), the evidence suggests that blind matching may 
serve as a useful stimulus to manage peer-learning, encouraging participants to seek out common 
ground in their pedagogic approaches and tutoring techniques, rather than concentrate on the 
familiar territory of teaching content and disciplinary issues. 

This study also underlines the importance of relationship building (contract negotiation and the 
search for common ground) between participants at the outset, as well as some form of closure 
activity at the end of the observation process (clarifying ‘meaning’ and discussing emergent 
themes of tutoring practice) — whichever matching technique is used or working process 
subsequently adopted. The evidence suggests that relationship building is a necessary first step to 
help create the conditions for a free-ranging and critical engagement to develop, which may 
endure beyond the formal timelines of the official observation. 

“Peer observation is best as part of an ongoing process — it creates the basis for future 
discussions. The value is not completely contained in the interaction at the time — it goes on 
beyond that. ” (Public Polity and Management tutor — Model 1) 

The guidance framework which has emerged from this study suggests a possible way of managing 
the peer exchange process, by embedding relationship building components into a structured 
step-by-step programme to support remote online participants. Future research may seek to test 
this framework with a wider set of online tutors to judge its suitability in creating the conditions 
for effective peer exchange. Any future study should also explore the value of the framework for 
both self-selecting and blind-matched tutors to judge whether participant control on the selection 
of pairings has any bearing on the relationship building process. 
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